Papers with linear models
Evaluating Composition Models for Verb Phrase Elliptical Sentence Embeddings (N19-1)
Copied to clipboard
| Challenge: | ellipsis is a natural language phenomenon where part of a sentence is missing and its information must be recovered from its context. |
| Approach: | They develop models for embedding VP-elliptical sentences using word embeddments . they extend existing verb disambiguation and sentence similarity datasets to elliptic phrases . |
| Outcome: | The proposed models outperform existing models on verb disambiguation and sentence similarity datasets and their linear counterparts. |
Classical Structured Prediction Losses for Sequence to Sequence Learning (N18-1)
Copied to clipboard
| Challenge: | Recent work on training neural attention models at the sequence level has focused on a series of objective functions commonly used for structured prediction. |
| Approach: | They propose to use objective functions commonly used to train linear models for structured prediction to train neural attention models at the sequence-level using either reinforcement learning-style methods or beam search optimization. |
| Outcome: | The proposed model outperforms beam search optimization on German-English translation and abstractive summarization tasks. |
Dynamic Meta-Metrics: Source-Sentence Conditioned Weighting for MT Evaluation (2026.acl-srw)
Copied to clipboard
| Challenge: | Rather than relying on a single static ensemble or language-specific weighting, DMM adapts the metric combination based on properties of the source segment. |
| Approach: | They propose a framework for machine translation evaluation that learns source-sentence conditioned combinations of existing metrics. |
| Outcome: | The proposed framework outperforms linear and Gaussian process-based ensembles across multiple language pairs and introducing soft conditioning yields gains over linear models. |
Linguistic Knowledge and Transferability of Contextual Representations (N19-1)
Copied to clipboard
| Challenge: | Recent work has explored contextual word representations, which assign each word a vector that is a function of the entire input sequence. |
| Approach: | They compare pretrained word representations with 16 diverse probing tasks to examine their transferability. |
| Outcome: | The pretrained representations are successful across a diverse set of NLP tasks . the models are competitive with state-of-the-art models but fail on fine-grained tasks requiring fine-granular knowledge, the study finds . |
ModRWKV: Transformer Multimodality in Linear Time (2025.emnlp-main)
Copied to clipboard
| Challenge: | Currently, multimodal studies are based on large language models with quadratic-complexity Transformer architectures. |
| Approach: | They propose a decoupled multimodal framework built upon the RWKV7 architecture as its LLM backbone and a lightweight architecture to achieve multi-source information fusion. |
| Outcome: | The proposed framework achieves multi-source information fusion through dynamically adaptable heterogeneous modality encoders. |
Deconvolutional Time Series Regression: A Technique for Modeling Temporally Diffuse Effects (D18-1)
Copied to clipboard
| Challenge: | a confound exists in time series data that violates assumptions of linear models . time series may violate assumptions through temporal diffusion . |
| Approach: | They propose a statistical model that borrows from digital signal processing to fit latent impulse response functions of arbitrary shape. |
| Outcome: | The proposed model recovers true latent IRFs and improves prediction quality . it is based on a new technique that borrows from digital signal processing . |
Can LLMs Find a Needle in a Haystack? A Look at Anomaly Detection Language Modeling (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Anomaly detection (AD) is a problem in machine learning, but it is not always competitive on certain datasets. |
| Approach: | They propose a new approach to Anomaly detection based on large pre-trained language models in three modalities. |
| Outcome: | The proposed model beats baselines on anomaly detection when presented as imbalanced classification problem regardless of the concentration of anomalous samples. |
Learning Only from Relevant Keywords and Unlabeled Documents (D19-1)
Copied to clipboard
| Challenge: | Existing methods for document classification are limited due to labeling and privacy concerns. |
| Approach: | They propose a super-vised text classification framework that provides keywords as a hint for classifying a document to a target class. |
| Outcome: | The proposed framework is simple to implement and has flexible choices of models, e.g., linear models or neural networks. |
Automated Molecular Concept Generation and Labeling with Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Concept-based models lack explainability and need predefined concepts and manual labeling in molecular science. |
| Approach: | They propose a framework that leverages Large Language Models to generate and label predictive molecular concepts without human input. |
| Outcome: | The proposed framework outperforms existing models on several benchmarks while maintaining explainability and allowing easy intervention. |
TuckER: Tensor Factorization for Knowledge Graph Completion (D19-1)
Copied to clipboard
| Challenge: | Knowledge graphs contain only a small subset of all possible facts . link prediction is a task of inferring missing facts based on existing facts - knowledge graphs are expensive and lack of information is needed to add new information. |
| Approach: | They propose a linear model based on Tucker decomposition of knowledge graph triples . they show that the model is expressive and has sufficient bounds on its embedding dimensionalities . |
| Outcome: | The proposed model outperforms state-of-the-art models across standard datasets and acts as a strong baseline for more elaborate models. |
On the Role of Pre-trained Language Models in Word Ordering: A Case Study with BART (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work uses linear models and neural networks for word ordering, yet pre-trained language models have not been studied in word ordering. |
| Approach: | They propose a constrained language generation task using unordered words as input. |
| Outcome: | The proposed model is able to perform better than existing models and proves to be reliable. |
CHiLL: Zero-shot Custom Interpretable Feature Extraction from Clinical Notes with Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study suggests that linear models with interpretable features are more reliable than opaque models. |
| Approach: | They propose an approach for natural-language specification of features for linear models . they prompt LLMs with expert-crafted queries to generate interpretable features from health records . |
| Outcome: | The proposed approach can be used to craft features clinically meaningful for downstream tasks . it is based on a risk prediction task and standard predictive tasks based upon this data . |
What Do Vision–Language Models Encode for Personalized Image Aesthetics Assessment? (2026.findings-acl)
Copied to clipboard
| Challenge: | Personalized image aesthetics assessment (PIAA) is an important research problem with practical applications. |
| Approach: | They propose a vision-language model that encodes multi-level aesthetic attributes . they analyze visual representations of VLMs to examine their internal representations . |
| Outcome: | The proposed framework can be used to personalize images without fine-tuning . it can be implemented in a variety of image domains and architectures. |